A Kaczmarz Method for Low Rank Matrix Recovery

The Kaczmarz method [1], [2], [3] was initially proposed as a row-based technique for reconstructing signals by finding the solutions to overdetermined linear systems. Its usefulness has seen wide application in irregular sampling and tomography [4], [5], [6]. In recent years, several modifications to the Kaczmarz update iterations have improved the recovery capabilities [7], [8], [9], [10], [11]. In particular, signal sparsity was exploited in [12], [13] and low-rankness in [14] to improve the rate of convergence in the overdetermined case while also enabling recovery from underdetermined linear systems. Consider the linear system of equations b = A(X), where X ∈ Rm×n is a rank r matrix of size m × n with r min{m,n}, A : Rm×n → p is a linear operator that samples p measurements from X, and b ∈ R is a measurement vector. The linear system above can be written in vector form as b = Ax, where x ∈ R is a vectorization of X, and A = [a1a2 . . .ap] : R → R is the corresponding measurement matrix with rows ai for i ∈ {1 . . . p}. Given an estimate xt of the signal x at iteration t, the Kaczmarz method proceeds by projecting xt orthogonally onto the solution space defined by row i of A, i.e.,